The present invention relates to an image information encoding apparatus, an image information decoding apparatus, and an image information recording and reproducing system for use with a digital VCR (video cassette recorder), and a disk drive, and an edit system using such devices.
FIGS. 1A and 1B of the accompanying drawings show a digital VCR as an example of an apparatus using a video encoder and a video decoder conforming to storage moving image encoding standard standardized based on standardization promoted according to the MPEG (moving picture image coding experts group). FIG. 1A shows, in block form, a recording system of digital VCR, and FIG. 1B shows, in block form, a reproducing system thereof.
As shown in FIG. 1A, the recording system of the digital VCR comprises an interface circuit 2 for converting an analog signal, supplied thereto through an input terminal 1 from a signal source (not shown), into digital data, an encoder 3 for encoding the digital data supplied thereto from the interface circuit 2, a shuffling circuit 4 for shuffling encoded output from the encoder 3 such that the encoded output is rearranged from the original data arrangement, an ECC (error correction code) encoder 5 for generating data string in the form of a product code by adding inner parity and outer parity to the output from the shuffling circuit 4, a channel encoder 6 for effecting digital modulation on the output from the ECC encoder 5, a recording amplifier 7 for amplifying in current the output from the channel encoder 6 and supplying the amplified output to recording heads 8, and a plurality of recording heads 8 mounted on a rotary drum (not shown) at a predetermined angular spacing.
The encoder 3 comprises a video encoder 3v for encoding video data of the digital data supplied thereto from the interface circuit 2 in a manner conforming to the above storage moving image coding standard, and an audio encoder 3a for encoding audio data in the digital data supplied thereto from the interface circuit 2.
An operation of the recording system of the digital VCR shown in FIG. 1A will be described.
The analog signal from the signal source, not shown, is supplied to the interface circuit 2, and converted into digital data by the interface circuit 2. The digital data from the interface circuit 2 is supplied to the encoder 3.
Of the digital data supplied to the encoder 3, the video data is supplied to the video encoder 3v, and encoded by the video encoder 3v. Of the digital data supplied to the encoder 3, the audio data is supplied to the audio encoder 3a, and encoded by the audio encoder 3a.
The video data encoded by the video encoder 3v and the audio data encoded by the audio encoder 3a are both supplied to the shuffling circuit 4. The video and audio data supplied to the shuffling circuit 4 are rearranged such that data arrangements thereof become different from those obtained when they are inputted to the recording system. The video data and audio data shuffled by the shuffling circuit 4 are sequentially supplied to the ECC encoder 5. The video and audio data supplied to the ECC encoder 5 are added with the inner parity and the outer parity, and thereby converted in the form of data string of product code. The video and audio data that had been converted in the form of data string of product code by the ECC encoder 5 are further added with decode information EDa supplied from the encoder 3, and supplied to and digitally modulated by the channel encoder 6. The video and audio data thus digitally modulated by the channel encoder 6 are supplied through the recording amplifying circuit 7 to the recording heads 8, and recorded on the magnetic tape 9 by the recording heads 8 so as to form slant tracks.
As shown in FIG. 1B, the reproducing system of the digital VCR comprises a plurality of recording heads 10 mounted on a rotary drum (not shown) at a predetermined angular spacing, a playback amplifying circuit 11 for amplifying playback RF signal reproduced by these reproducing heads 10, a waveform equalizing circuit 12 for equalizing a waveform of the playback RF signal by phase-correcting and amplitude-correcting the playback RF signal supplied thereto from the playback amplifying circuit 11, a channel decoder 13 for decoding the output from the waveform equalizing circuit 12, an ECC decoder 14 for error-correcting video data in the decoded output from the channel decoder 13 by the inner parity and the outer parity added to the video data, adding an error flag for error-concealing uncorrectable error by some suitable methods such as interpolation, error-correcting the audio data by use of the inner parity and the outer parity added to the audio data, and adding error flag for error-concealing uncorrectable error by some proper methods such as interpolation, a de-shuffling circuit 15 for reconverting data strings of the video and audio data supplied thereto from the ECC decoder 15 into original data strings obtained before these data are shuffled by the shuffling circuit 4 upon recording, a decoder 16 for decoding the video and audio data supplied thereto from the de-shuffling circuit 15 based on the decode information DDa from the ECC decoder 14 to provide original video and audio data, an error concealing circuit 17 for error-concealing the video data when the error flag is added to the video data from the decoder 16, and error-concealing the audio data when the error flag is added to the audio data from the decoder 16, and an interface circuit 18 for converting the video and audio data supplied thereto from the error-concealing circuit 17 into analog video signals, and outputting the analog video and audio signals through an output terminal 19.
The decoder 16 comprises a video decoder 16v for decoding the video data supplied thereto from the de-shuffling circuit 15 in a manner conforming to the storage moving image coding standard, and an audio decoder 16a for decoding the audio data supplied thereto from the de-shuffling circuit 15. While this reproducing system includes only one output terminal 19 in FIG. 1B, in actual practice, the output terminal 19 comprises a video signal output terminal and an audio signal output terminal although not shown.
An operation of the reproducing system of the digital VCR shown in FIG. 1B will be described.
Record data that has been recorded on the magnetic tape by the recording heads 8 so as to form oblique tracks is reproduced by the reproducing heads 10, and supplied through the playback amplifying circuit 11 to the waveform-equalizing circuit 12 as the playback RF signal. The playback RF signal supplied to the waveform-equalizing circuit 12 is waveform-equalized by phase-correction and amplitude-correction, and supplied to the channel decoder 13, in which it is demodulated. The video and audio data from the channel decoder 13 are supplied to the ECC decoder 14, and the inner parities and the outer parities added to the video and audio data are extracted by the ECC decoder 14. Then, the ECC decoder 14 error-corrects the video and audio data by the inner parities and outer parities thus extracted, and adds the error flags indicative of uncorrectable data to uncorrectable video and audio data.
The video and audio data from the ECC decoder 14 are supplied to the de-shuffling circuit 15, and reconverted by the de-shuffling circuit 15 into data arrangement obtained before the video and audio data are shuffled by the shuffling circuit 4 upon recording. The video and audio data from the de-shuffling circuit 15 are supplied to the decoder 16, and the video decoder 16v in the decoder 16 decodes the video data supplied to the decoder 16 based on the decode information DDA from the ECC decoder 14 to provide the original video data. The audio decoder 16a in the decoder 16 decodes the audio data supplied to the decoder 16 to provide the original audio data. The video and audio data from the decoder 16 are supplied to the error concealing circuit 17, and the error concealing circuit 17 error-conceals only video and audio data with the error flags added thereto. The video and audio data from the error concealing circuit 17 are supplied to and converted by the interface circuit 18 into analog video and audio signals, and outputted through the output terminal 19. When a television monitor is connected to the video signal output terminal of the output terminal 19 and a speaker is connected through an amplifier to an audio signal output terminal of the output terminal 19, the video signal is reproduced on a picture screen of the television monitor as an image, and the audio signal is emanated from the speaker as sounds.
The video encoder 3v shown in FIG. 1A will be described more fully with reference to FIG. 2. FIG. 2 shows, in block form, an inside arrangement of the video encoder 3v shown in FIG. 1A.
In the video encoder 3v, as shown in FIG. 2, the video data is supplied to an input terminal 20 from the interface circuit 2 shown in FIG. 1A. The input terminal 20 is connected to a first input terminal of a motion detecting circuit 21, the other input terminal of a motion compensating circuit 24 and an input terminal of a frame memory 22. An output terminal of the frame memory 22 is connected to a second input terminal of the motion detecting circuit 21, an input terminal of the frame memory 22, an adding-side input terminal of an adding circuit 27, an intra-side fixed contact b of a switch 28, and the other input terminal of an inter/intra judgement circuit 29. An output terminal of a frame memory 23 is connected to a third input terminal of the motion detecting circuit 21 and the other input terminal of a motion compensating circuit 25. An output terminal of the motion compensating circuit 24 is connected to one adding-side input terminal of an adding circuit 26 which incorporates therein a 1/2-multiplier. An output terminal of the motion compensating circuit 25 is connected to the other adding-side input terminal of the adding circuit 26, and an output terminal of the adding circuit 26 is connected to a subtracting-side input terminal of the adding circuit 27, and an output terminal of the adding circuit 27 is connected to an inter-side fixed contact a of the switch 28, and one input terminal of an inter/intra judgement circuit 61. A movable contact c of the switch 28 is connected to an input terminal of a DCT (discrete cosine transform) circuit 30, and an output terminal of the DCT circuit 30 is connected to an input terminal of a quantizing circuit 31. An output terminal of the quantizing circuit 31 is connected to an input terminal of a variable-word length encoding circuit 32, and an output terminal of the variable-word length encoding circuit 32 is connected through an output terminal 33 to the input terminal of the shuffling circuit 4 shown in FIG. 1A. The output terminal of the motion detecting circuit 21 is connected to input terminals of the motion compensating circuits 24, 25, and the input terminal of the variable-word length encoding circuit 32, and the frame memories 22, 23, the inter/intra judgement circuit 29, and the output terminal 33 for outputting the decode information EDa are connected to a controller 35.
The frame memories 22, 23 read and write image data in response to read/write control signals R/W supplied thereto from the controller 35.
At the time frame image data is stored in the frame memory 22, if the output of the frame memory is a present frame, then the frame image data supplied to the input terminal 20 becomes a future frame, and frame image data stored in the frame memory 23 becomes a past frame. The present frame will be referred to as "present frame", the future frame will be referred to as "succeeding frame", and the past frame will be referred to as "preceding frame", hereinafter, respectively.
The motion detecting circuit 21 motion-detects the frame image data supplied thereto through the input terminal 20, the frame image data read out from the frame memory 22, and the frame image data read out from the frame memory 23 at the unit of macroblock comprising 16 lines.times.16 pixels, for example. As a motion detecting method, there is known a motion detecting method based on a so-called full-search block matching, for example.
The block matching obtains difference absolute value sum data between macroblock data MB (f) of macroblock data MB (f) of present frame and macroblock data MB (f-1) of all preceding frames provided within a search area set in the preceding frame, and obtains motion vector data based on difference absolute sum data of smallest value, address of macroblock data MB (f-1) of preceding frame where the difference absolute value sum data could be obtained, and address of macroblock data MB (f) of present frame. The resultant motion vector data indicates macroblock data MB (f-1) of preceding frame having an optimum level arrangement pattern, i.e., level arrangement pattern most close to the level arrangement pattern of macroblock data MB (f) of present frame for calculating a difference between the macroblock data MB (f-1) of preceding frame and the macroblock data MB (f) of present frame. The motion compensation processing extracts the macroblock data MB (f-1) of preceding frame indicated by the motion vector data from the preceding frame.
The motion detecting circuit 21 motion-detects the macroblock data MB (f) of present frame stored in the frame memory and the macroblock data (f+1) of succeeding frame supplied thereto through the input terminal 20 to provide motion vector data MV based on a motion-detected result. The motion detecting circuit 21 motion-detects the macroblock data of present frame stored in the frame memory 22 and the macroblock data MB (f-1) of preceding frame stored in the frame memory 23 to provide motion vector data MV based on a motion-detected result.
While the signal link connected to the output terminal of the motion detecting circuit 21 is shown by a single line and only one "MV" is referred to as a symbol indicating motion vector data in FIG. 2, the number of motion vector data is not limited thereto, and motion vector data of all macroblocks of frame image data stored in the frame memory can be provided each time motion detection is implemented.
The motion compensating circuit 24 extracts macroblock data MB (f+1) of contents closest to those of macroblock data MB (f) of present frame to be processed from the frame image data of succeeding frame supplied thereto through the input terminal 20 based on the motion vector data supplied thereto from the motion detecting circuit 21, and supplies the macroblock data MB (f+1) thus extracted to the adding circuit 26.
The motion compensating circuit 25 extracts macroblock data MB (f-1) of contents closest to those of macroblock data MB (f) of present frame to be processed from the frame image data of preceding frame stored in the frame memory 23 based on the motion vector data supplied thereto from the motion detecting circuit 21, and supplies the macroblock data MB (f-1) thus extracted to the adding circuit 26.
The adding circuit 26 adds the macroblock data MB (f+1) from the motion compensating circuit 24 and the macroblock data MB (f-1) from the motion compensating circuit 25, and multiplies a coefficient "1/2" to the added result by the 1/2-multiplier incorporated therein to provide mean value data between the macroblock data MB (f+1) from the motion compensating circuit 24 and the macroblock data (f-1) from the motion compensating circuit 25.
The adding circuit 27 subtracts the mean value data of the adding circuit 26 from the macroblock data MB (f) of present frame supplied thereto from the frame memory 22 to provide difference data between the macroblock data MB (f) of present frame and macroblock data obtained as mean value data by bidirectional prediction.
The inter/intra judgement circuit 29 properly allows the switch 28 to connect the movable contact c to the inter-side fixed contact a or the intra-side fixed contact b based on the difference data from the adding circuit 27, the macroblock data MB (f) from the frame memory 22, and a frame pulse Fp supplied thereto from the controller 35.
Summarily, the frame image data of present frame stored in the frame memory 22 is to be encoded, and the processing unit is the macroblock unit. The reason that the motion detecting circuit 21 implements the motion detection is to search the macroblock data MB (f+1) and MB (f-1) of succeeding and preceding frames closest to contents of the macroblock data MB (f) of present frame to be encoded. The detected results obtained when the above search is completed, or when the macroblock data MB (f+1) and MB (f-1) of succeeding and preceding frames closest to the contents of the macroblock data MB (f) of present frame are the motion vector data MV. When the macroblock data (f+1) and MB (f-1) of succeeding and preceding frames closest to the contents of the macroblock data MB (f) of present frame are extracted by use of the motion vector data MV, data having contents common to those of data that has been transmitted before can be inhibited from being transmitted.
However, because the macroblock data MB (f) of present frame subtracted from the macroblock data, obtained by bidirectional prediction, by the adding circuit 27 cannot be decoded by only the difference data upon decoding, as shown in FIG. 2, the motion vector data MV is supplied to the variable-word length encoding circuit 32, compressed by the variable-word length encoding circuit 32, and then transmitted together with the difference data.
A role that the inter/intra judgement circuit 29 plays is to select either the encoding of the difference data or the encoding of the output from the frame memory 22. To encode difference data, i.e., interframe difference information is referred to as "interframe encoding", and to encode the output from the frame memory as it is referred to as "intraframe encoding". The word "encoding" means not the difference calculation done by the adding circuit 27 but encoding executed by circuits, which will be described later on, following the DCT circuit 30. Although the inter/intra judgement circuit 29 switches the interframe encoding and the intraframe encoding at the macroblock unit, let it be assumed that the inter/intra judgement circuit 29 switches the interframe encoding and the intraframe encoding at the frame unit in order to understand the following description more clearly.
Image data of respective frames outputted from the switch 28 and which are to be encoded are generally referred to as "I picture", "B picture", and "P picture" in accordance with the encoded forms.
The I picture is one frame-encoded image data which results from intraframe-encoding the macroblock data MB (f) of present frame outputted from the switch 28. The word "encoding" used herein means "encoding" executed by the DCT circuit 30, the quantizing circuit 31 and the variable-word length encoding circuit 32. Accordingly, in the case of I picture, the movable contact c of the switch 28 is constantly connected to the fixed contact b under control of the inter/intra judgement circuit 29.
The P picture is one frame-encoded image data comprising data which results from encoding (interframe-encoding) difference data between macroblock data MB (f) of present frame outputted from the switch 28 and I or P picture motion-compensated macroblock data which becomes a preceding frame relative to the macroblock data MB (f) of present frame and data which results from intraframe-encoding the macroblock data MB (f) of present frame. However, when the P picture is generated, the motion vector data MV for motion-compensating image data serving as the I picture is obtained based on image data to be encoded as P picture and image data provided immediately before the above image data.
The B picture is data which results from encoding (interframe-encoding) the difference data between the macroblock data MB (f) of present frame outputted from the switch 28 and macroblock data of the following six kinds.
Of the macroblock data of six kinds, macroblock data of two kinds are the macroblock data MB (f) of present frame outputted from the switch 28 and I or P picture motioncompensated macroblock data which becomes the preceding frame relative to the macroblock data MB (f) of present frame. Of the macroblock data of six kinds, macroblock data of other two kinds are the macroblock data MB (f) of present frame outputted from the switch 28 and I or P picture motion-compensated macroblock data which becomes a preceding frame relative to the macroblock data MB (f) of present frame. Of the macroblock data of six kinds, macroblock data of other two kinds are interpolated macroblock data which are generated from I picture which becomes a preceding frame relative to the macroblock data MB (f) of present frame outputted from the switch 28 and P picture which becomes a preceding frame, and interpolated macroblock data generated from a P picture which becomes a preceding frame relative to the macroblock data MB (f) of present frame outputted from the switch 28 and a P picture which becomes a preceding frame.
As is clear from the above description, the P picture contains data which is encoded by use of image data of other frames than the present frame, i.e., interframe-encoded data. Also, the B picture is composed of only interframe-encoded data, and hence the B picture cannot be decoded by itself. Therefore, as is conventional, a plurality of pictures are collected as one GOP (Group Of Picture), and it is standardized that such image data is processed at the unit of GOP.
In general, the GOP comprises one or a plurality of I pictures and zero or a plurality of non-I pictures. In order to facilitate the description below, let it be assumed that intraframe-encoded data is I picture, bidirectionally-predicted and encoded image data is B picture, and that one GOP is composed of one B picture and one P picture.
As is clear from the above description, in FIG. 2, the I picture is generated through a route comprising the output of the frame memory 22, the switch 28, the DCT circuit 30, and circuits following the DCT circuit 30. Also, the B picture is generated through a route comprising the input terminal 20, the motion compensating circuit 24, the adding circuit 26, the output terminal of the frame memory 23, the motion compensating circuit 25, the adding circuit 26, the adding circuit 26, the adding circuit 27, the switch 28, the DCT circuit 30, and circuits following the DCT circuit 30.
The DCT circuit 30 transforms the output of the switch 28 in the form of coefficient data of DC to high-order AC component. The quantizing circuit 31 quantizes the coefficient data supplied thereto from the DCT circuit 30 at a predetermined quantization step size. The variable-word length encoding circuit 32 encodes the coefficient data supplied thereto from the quantizing circuit 31 and the motion vector data MV supplied thereto from the motion detecting circuit 21 by some proper methods such as Huffman coding or run-length-coding. When data is outputted, decode information, frame data of B picture, decode information, an frame data of I picture are arranged within one GOP, in that order.
The decode information EDa comprises GOP header data indicative of the head of GOP, an inter/intra selection signal SEL or the like. The GOP header data of "1" shows that frame data with the GOP header data added to the head thereof is frame data of the header of GOP, and the GOP header data of "0" shows that frame data with the GOP header data added to the head thereof is not the head of GOP but the head of picture.
An operation of the video encoder 3v will be described. When I picture composing one GOP is generated, the switch 28 connects the movable contact c to the intra-side fixed contact b under control of the inter/intra judgement circuit 29. In this case, the frame image data read out from the frame memory 22 is encoded by the DCT circuit 30 and circuits following the DCT circuit 30. At that time, the decode information EDa from the controller 35 is supplied through the output terminal 34 to the ECC encoder 5 shown in FIG. 1A.
Subsequently, when B picture composing one GOP is generated, the switch 28 connects the movable contact c to the inter-side fixed contact a under control of the inter/intra judgement circuit 29.
The motion detecting circuit 21 motion-detects image data by the macroblock data MB (f) of present frame and the macroblock data MB (f+1) of succeeding frame provided within the frame image data sequentially. As a consequence, the motion detecting circuit 21 selects the macroblock data MB (f+1) with contents best agreed with those of the macroblock data MB (f) of present frame, and obtains motion vector data MV indicative of the position of the macroblock data MB (f+1) where the position of the macroblock data MB (f) of present frame is assumed to be a starting point. Similarly, the motion detecting circuit 21 motion-detects image data by the macroblock data MB (f) of present frame and the macroblock data MB (f-1) of preceding frame provided within the frame image data sequentially. As a consequence, the motion detecting circuit 21 selects the macroblock data MB (f-1) with contents best agreed with those of the macroblock data MB of present frame, and obtains motion vector data MV indicative of the position of the macroblock data MB (f-1) where the position of the macroblock data of present frame is assumed to be a starting point.
The above two motion vector data MV are supplied to the variable-word length encoding circuit 32, and the motion compensating circuits 24, 25, respectively. The motion compensating circuit 24 extracts the macroblock data MB (f+1), shown by the motion vector data MV, from the frame image data of succeeding frame, and the macroblock data MB (f+1) thus extracted is supplied to the adding circuit 26. On the other hand, the motion compensating circuit 25 extracts the macroblock data MB (f-1), shown by the motion vector data MV, from the frame image data of preceding frame, and the macroblock data MB (f-1) thus extracted is supplied to the adding circuit 26.
The adding circuit 26 adds the macroblock data MB (f+1) from the motion compensating circuit 24 and the macroblock data MB (f-1) from the motion compensating circuit 25, and averages the added result with a multiplication of the coefficient "1/2" to the added result to provide mean value data. The mean value data is supplied through the subtracting-side input terminal of the adding circuit 27 to the adding circuit 27. The macroblock data MB (f) of present frame read out from the frame memory 22 is supplied to the adding-side input terminal of the adding circuit 27. Thus, the adding circuit 27 subtracts the mean value data supplied thereto from the adding circuit 26 from the macroblock data MB (f) of present frame. The output from the adding circuit 27 is interframe-encoded by the DCT circuit 30, the quantizing circuit 31, and the variable-word length encoding circuit 32, and outputted as B picture.
After all macroblock data MB (f) stored in the frame memory 22 have been processed in the above manner, i.e., interframe-encoded, the frame image data is read out from the frame memory 22, and supplied to the frame memory 23, in which it is stored as image data of preceding frame. On the other hand, image data of the next frame is stored in the frame memory 22 as frame image data of present frame. At that very time, the decode information from the controller 35 is supplied through the output terminal 34 to the ECC encoder 5 shown in FIG. 1A.
A concept in which image data is encoded by the video encoder 3v will be described with reference to FIG. 3. FIG. 3 is a conceptual diagram used to explain a manner in which the video encoder 3v encodes image data.
FIG. 3 shows frame image data to be encoded, and frame numbers F1 through F10 are illustrated on the lower portions of these encoded image data. Frame image data shown hatched indicate frame image data I1, I3, I5, I7 and I9 serving as I pictures, and frame image data that are not shown hatched indicate frame image data B2, B4, B6, B8, B10 serving as B pictures (or frame image data P2, P4, P6, P8, P10 serving as P pictures). In this case, the frame image data I1 and B2 of the frame numbers F1 and F2 constitute one GOP; the frame image data I3 and B4 of the frame numbers F3 and F4 constitute one GOP; the frame image data I5 and B6 of the frame numbers F5 and F6 constitute one GOP; frame image data I7 and B8 of the frame numbers F7 and F8 constitute one GOP; and frame image data I9 and B10 of the frame numbers F9 and F10 constitute one GOP.
Of the frame image data shown in FIG. 3, the frame image data I1, I3, I5, I7 and I9 of the frame numbers F1, F3, F5, F7, F9 are read out from the frame memory 22, supplied through the switch 28 to the DCT circuit 30, the quantizing circuit 31, and the variable-word length encoding circuit 32, in that order, and respectively intraframe-encoded by these circuits 30, 31, 32 as earlier noted.
When on the other hand image data of B pictures are encoded, frame image data provided at left and right of encoded frame image data, i.e., frame image data of preceding and succeeding frames are used as shown by arrows in FIG. 3, and the encoded image data is interframe-encoded. By way of example, the frame image data I1 and I3 of preceding and succeeding frames are used for the frame image data of the frame number F2, and the frame image data I3 and I5 of preceding and succeeding frames are used for the frame image data of the frame number F4.
When the frame image data B2 of the frame number F2, for example, is encoded, the frame image data B2 of the frame number F2 is stored in the frame memory 22 shown in FIG. 2 as frame image data of present frame. At that time, the frame image data I1 of the frame number F1 is stored in the frame memory 23 as frame image data of preceding frame. Then, when the processing is started, the frame image data I3 of the frame number F3 is supplied through the input terminal 20 as frame image data of succeeding frame.
The motion detecting circuit 21 motion-detects image data by the macroblock data MB (f) of the frame image data B2 of the frame number F2 and the macroblock data MB (f-1) of the frame image data I2 of the frame number F1 read out from the frame memory 23 to provide one motion vector data MV. Also, the motion detecting circuit 21 motion-detects image data by the macroblock data MB (f) of the frame image data B2 of the frame number F2 and the macroblock data MB (f+1) of the frame image data I3 of the frame number F3 supplied thereto through the input terminal 20 to provide one motion vector data MV.
The motion compensating circuit 24 extracts the macroblock data MB (f-1) of the frame image data I1 of the frame number F1 shown by the motion vector data MV. The motion compensating circuit 25 extracts the macroblock data MB (f-1) of the frame image data I3 of the frame number F3 shown by the motion vector data MV. The macroblock data MB (f-1) and MB (f+1) extracted by the motion compensating circuits 24 and 25 are closest to the contents, i.e., the level arrangement of pixel data within the macroblock of the macroblock data (f) of the frame image data B2 of the frame number F2.
The adding circuit 26 adds the macroblock data MB (f-1) in the frame image data I1 of the frame number F1 supplied thereto from the motion compensating circuit 24 and the macroblock data MB (f+1) in the frame image data I3 of the frame number F3 supplied thereto from the motion compensating circuit 25, and adds the coefficient "1/2" to the added result by the 1/2-multiplier incorporated therein to provide mean value data of the two macroblock data MB (f-1) and MB (f+1). This mean value data is supplied through the subtraction-side input terminal of the adding circuit 27 to the adding circuit 27.
On the other hand, macroblock data MB (f-1) of the frame image data B2 of the frame number F2 is supplied through the adding-side input terminal of the adding circuit 27 to the adding circuit 27. Therefore, the adding circuit 27 subtracts the mean value data from the macroblock data MB (f) of the frame image data B2 of the frame number F2 to provide difference data. The resultant difference data is supplied through the switch 28 to the DCT circuit 30 and the circuits following the DCT circuit 30, and thereby encoded. Then, all macroblock data MB (f) of the frame image data B2 of the frame number 2 are processed in the above-mentioned manner to interframe-encode the frame image data B2 of the frame number F2. The frame image data B4, B6, B8 and B10 of the frame numbers F4, F6, F8 and F10 also are interframe-encoded by similar processing.
Referring to FIG. 4, the video decoder 16v shown in FIG. 1B will be described. FIG. 4 shows, in block form, an arrangement of the video decoder 16v shown in FIG. 1B.
As shown in FIG. 4, the video decoder 16v comprises a variable-word length decoding circuit 42 for decoding reproduced image data, supplied thereto from the de-shuffling circuit 15 shown in FIG. 1B through an input terminal 40, to provide quantized coefficient data, decoding motion vector data MV and supplying the decoded motion vector data MV to motion compensating circuits 47, 48 which will be described later on, an inverse quantizing circuit 43 for inverse-quantizing the output of the variable-word length decoding circuit 42 to provide coefficient data transformed by DCT, an IDCT (inverse discrete cosine transform) circuit 44 for inverse-discrete-cosine-transforming the output of the inverse quantizing circuit 43 to provide image data of I or B picture, a frame memory 45 for storing the output of the IDCT circuit 44 in response to a read/write control signal R/W supplied thereto from a controller 53, a frame memory 46 for storing image data read out from the frame memory 45 in response to a read/write control signal R/W supplied thereto from the controller 53, a motion compensating circuit 47 for extracting macroblock data indicated by motion vector data MV from the variable-word length encoding circuit 42 from the frame image data supplied thereto as I picture from the IDCT circuit 44, a motion compensating circuit 48 for extracting macroblock data shown by the motion vector data Mv supplied thereto from the variable-word length encoding circuit 42 from the frame image data stored in the frame memory 46, an adding circuit 49 for adding the macroblock data from the motion compensating circuit 47 and the macroblock data supplied from the motion compensating circuit 48 and multiplying coefficient "1/2" to the added result by a 1/2-multiplier incorporated therein to provide mean value data of the two macroblock data, an adding circuit 50 for adding the mean value data of the adding circuit 49 and the difference data serving as B picture read out from the frame memory 45 to provide the original macroblock data, and a switch 51 for switching the macroblock data from the adding circuit 50 and the macroblock data read out from the frame memory 49 based on an inter/intra selection signal SEL supplied thereto from the controller 53, and supplying a switched output through an output terminal 52 to the error concealing circuit 17 shown in FIG. 1B as reproduced video data.
In FIG. 4, data of I picture is decoded through a data path of an output of the frame memory 45 and the switch 51. Also, data of B picture is decoded through a data path of the IDCT circuit 44, the motion compensating circuit 47 and the adding circuit 49, the output terminal of the frame memory 46, the motion compensating circuit 48 and the adding circuit 49, the adding circuit 50 and the switch 51.
An operation of the video decoder 16v shown in FIG. 4 will be described. To facilitate the explanation, let it be assumed that, upon decoding, image data serving as decoded I picture of immediately-preceding GOP is stored in the frame memory 46, image data serving as B picture of GOP to be decoded is stored in the frame memory 45, and that image data serving as decoded I picture of GOP to be decoded is outputted from the IDCT circuit 44. Moreover, the frame image data of decoded I picture of immediately-preceding GOP is frame image data of preceding frame, image data of B picture of GOP to be decoded is difference image data of present frame, and frame image data of decoded I picture of GOP to be decoded is frame image data of succeeding frame.
The video data from the de-shuffling circuit 15 shown in FIG. 1B is supplied through the input terminal 40 shown in FIG. 4 to the variable-word length decoding circuit 42. On the other hand, the decode information DDa from the ECC decoder 14 shown in FIG. 1B is supplied through the input terminal 41 shown in FIG. 4 to the controller 53. The video data is decoded into quantized coefficient data of the variable-word length decoding circuit 42. The video data thus decoded is supplied to the inverse quantizing circuit 43, and reconverted by the inverse quantizing circuit 43 into coefficient data transformed by DCT. The video data that has been reconverted to the DCT coefficient data is supplied to the IDCT circuit 44, and thereby reconverted into the original image data. The "original image data" is "difference data" if this image data is B picture generated by interframe-encoding, and "macroblock data" if this image data is I picture generated by intraframe-encoding.
The controller 53 extracts GOP header data and the inter/intra selection signal SEL from the decode information DDa, and supplies the inter/intra selection signal SEL to the switch 51 as the switching control signal, whereby a movable contact c of the switch 51 is connected to an inter-side fixed contact a. Information indicating a timing at which the switch 51 is changed-over based on the inter/intra selection signal SEL is GOP header data detected from the decode information DDa. Since the GOP header data is added to every GOP, it is possible to determine that the period of one GOP data is from the detection of GOP header data of the detection of the next GOP header data.
The motion compensating circuit 47 extracts the macroblock data MB (f+1) shown by the motion vector data MV supplied thereto from the variable-word length decoding circuit 42 from the frame image data of succeeding frame serving as decoded I picture from the IDCT circuit 44 after the difference image data serving as B picture has been stored in the frame memory 45. The macroblock data MB (f+1) extracted from the frame image data serving as the succeeding frame by the motion compensating circuit 47 is supplied to the adding circuit 49.
On the other hand, the motion compensating circuit 48 starts the processing in unison with the processing start timing of the motion compensating circuit 47 after the frame image data serving as encoded I picture of immediately-preceding GOP has been stored in the frame memory 46. Specifically, the motion compensating circuit 48 extracts the macroblock data MB (f-1)O shown by the motion vector data MV supplied thereto from the variable-word length decoding circuit 42 from the frame image data serving as decoded I picture of immediately-preceding GOP. The macroblock data MB (f-1) extracted from the frame image data serving as the preceding frame by the motion compensating circuit 48 is supplied to the adding circuit 49.
Although the signal line connected to the output terminal of the variable-word length decoding circuit 42 is shown by a single line and one "MV" is used as reference symbol for indicating motion vector, motion vector data MV of all macroblocks of one frame image data are respectively obtained in each motion detection of the encoding processing as earlier noted with reference to FIGS. 1A, 1B. Accordingly, the motion vector data MV supplied to the motion compensating circuit 47 and the motion vector data MV supplied to the motion compensating circuit 48 are different motion vector data obtained upon encoding.
The macroblock data MB (f+1) from the motion compensating circuit 47 and the macroblock data MB (f-1) from the motion compensating circuit 48 are added by the adding circuit 49. The added result is multiplied with the coefficient "1/2" by the 1/2-multiplier incorporated within the adding circuit 49, and thereby averaged. Mean value data from the adding circuit 49 is supplied to the adding circuit 50.
The adding circuit 50 adds the difference data read out from the frame memory 45 and the mean value data from the adding circuit 49. The added output from the adding circuit 50 is supplied through the output terminal 52 to the error concealing circuit 17 shown in FIG. 1B as decoded reproduced video data of the present frame.
The read/write control signals R/W from the controller 53 are supplied to the frame memories 45, 46, whereby the macroblock data of succeeding frame serving as I picture outputted from the IDCT circuit 44 is supplied to the motion compensating circuit 47 and the frame memory 46, and thereby stored in the frame memory 46.
Meaning of the processing executed by the adding circuits 49, 50 will be described below one more time. The adding circuit 27 shown in FIG. 2 obtains the mean value between the macroblock data MB (f+1) of the succeeding frame obtained by the motion-compensation in the motion compensating circuit 24 and the macroblock data MB (f-1) of the preceding frame obtained by the motion-compensation in the motion compensating circuit 25 by calculation. Then, the adding circuit 27 subtracts the mean value data supplied thereto from the adding circuit 26 from the macroblock data MB (f) of the present frame, which is expressed by the following equation (1):
Difference data MBd (f) of macroblock unit of present frame EQU =MB(f)-{MB(f+1)+MB(f-1)}/2! (1)
Accordingly, in order to obtain the macroblock data MB (f) of the present frame from the difference data MBd, the video decoder 16v side has to carry out the calculation expressed by the following equation (2): EQU Macroblock data MB(f) of present frame=Difference data MBd(f) of macroblock unit of the present frame+{MB(f+1)+MB(f-1)}/2! (2)
Reference symbol "+", i.e., "addition" in the beginning of the equation (2) corresponds to addition executed by the adding circuit 50 shown in FIG. 4. Reference symbol "+" which shows addition of the macroblock data MB (f+1) and the macroblock data MB (f-1) in the parentheses in the equation (2) corresponds to an addition executed by the adding circuit 49 shown in FIG. 4. Furthermore, the coefficient "1/2" multiplied with data provided in the braces in the equation (2) corresponds to a multiplication of the coefficient "1/2".
In order to perform the calculation expressed by the equation (2), the macroblock data MB (f+1) should be extracted from the frame image data of succeeding frame thus transmitted, and the macroblock data MB (f-1) should be extracted from the frame image data of preceding frame thus transmitted. The motion vector data MV that have been supplied from the variable-word length decoding circuit 42 to the motion compensating circuits 47, 48 are used in the above-mentioned "extraction", respectively.
After all frame image data of the present frame serving as B pictures are decoded by repeatedly executing the above processings, the controller 53 supplies the inter/intra selection signal SEL to the switch 51, whereby the movable contact c of the switch 51 is connected to the inter-side fixed contact b. Since the contents of the frame memory 45 are rewritten as the macroblock data (f+1) of succeeding frame serving as I pictures in each processing, at this timing point, frame image data of succeeding frame serving as I picture is stored in the frame memory 45.
The read/write control signals R/W is supplied to the frame memory 45 from the controller 35, whereby the frame image data of succeeding frame serving as I picture is supplied through the switch 51 and the output terminal 52 to the error concealing circuit 17 shown in FIG. 1B.
Referring back to FIG. 3, a concept in which the video decoder 16v decodes image data will be described below. FIG. 3 is a conceptual diagram used to explain a decode processing executed by the video decoder 16v. FIG. 3 shows frame image data to be decoded, and the frame numbers are illustrated on the lower portions of these encoded image data, respectively. Frame image data shown hatched show frame image data serving as I pictures, and frame image data, not shown hatched, show frame image data serving as B pictures (or frame image data serving as P pictures), respectively.
Of the frame image data shown in FIG. 3, the frame image data I1, I3, I5, I7 and I9 of the frame numbers F1, F3, F5, F7 and F9 are respectively supplied to the variable-word length decoding circuit 42, the inverse quantizing circuit 43, and the IDCT circuit 44, in that order, as earlier noted, and decoded, respectively. Thereafter, these image data are stored in the frame memory 45, read out from the frame memory 45, and outputted through the switch 51 and the output terminal 52 as reproduced image data, respectively.
On the other hand, as shown by the arrows in FIG. 3, the frame image data serving as B picture is decoded by use of frame image data of left and right, i.e., preceding and succeeding frames. The frame image data B2 of the frame number F2, for example, is decoded by use of the frame image data I1 of the preceding frame and the frame image data I3 of the succeeding frame.
When the frame image data B2 of the frame number F2 is decoded, the frame image data B2 of the frame number F2 is stored in the frame memory 45 shown in FIG. 4 as the frame image data B2 of the present frame. At that time, the frame image data I1 of the frame number F1 is stored in the frame memory 46 as the frame image data of the preceding frame. Then, when the decode processing is started, the frame image data I3 of the frame number F3 is outputted from the IDCT circuit 44 as the frame image data of the succeeding frame.
The motion compensating circuit 47 extracts the macroblock data MB (f-1) from the image data I1 of the frame number F1 shown by the motion vector data MV from the variable-word length decoding circuit 42, The motion compensating circuit 48 extracts the macroblock data MB (f+1) of the frame image data I3 of the frame number F3 shown by the motion vector data MV from the variable-word length decoding circuit 42. The macroblock data MB (f-1) and MB (f+1) extracted by the motion compensating circuits 47, 48 are extracted and used as those data whose contents, i.e., level arrangements of pixel data in the macroblocks are closest to that of the macroblock data MB (f) of the frame image data B2 of the frame number F2.
The adding circuit 49 adds the macroblock data MB (f-1) in the frame image data I1 of the frame number F1 supplied thereto from the motion compensating circuit 47 and the data MB (f+1) in the frame image data I3 of the frame number F3 supplied thereto from the motion compensating circuit 48, and multiplies the added result with the coefficient "1/2" by the 1/2-multiplier incorporated therein to provide mean value data of the two macroblock data MB (f-1) and MB (f+1). The resultant mean value data is supplied through one addition-side input terminal of the adding circuit 50 to the adding circuit 50.
On the other hand, the macroblock data MB (f) in the frame image data B2 of the frame number F2 is supplied through the other addition-side input terminal of the adding circuit 50 to the adding circuit 50. Therefore, the adding circuit 50 adds difference data of the macroblock data MB (f) in the frame image data B2 of the frame number F2 to the mean value data to provide the original macroblock data MB (f). The macroblock data thus obtained by this calculation is outputted through the switch 51 and the output terminal 52. The above-mentioned processings are effected on all macroblock data MB (f) of the frame image data B2 of the frame number F2, and thereby the frame image data B2 of the frame number F2 is decoded. The frame image data B4, B6, B8 and B10 of the frame numbers F4, F6, F8 and F10 also are decoded by the similar processings.
U.S. Pat. No. 4,897,720 describes the above fullsearch block matching. U.S. Pat. No. 4,969,055 describes GOP, and U.S. Pat. No. 4,383,272 describes B picture. U.S. Pat. No. 4,232,338 describes P picture, and U.S. Pat. No. 4,894,713 describes encoding based on DCT, quantization and variable-word length encoding.
Recording and reproducing apparatus such as VCR are generally able to reproduce recorded data recorded on a recording medium in the five modes, in total, such as normal-speed positive direction playback, normal-speed reverse direction playback, positive direction playback at other speed than the normal speed, reverse direction playback at other speed than the normal speed, and still playback.
The digital VCR including the video encoder for encoding image data and the video decoder for decoding image data encoded by the video encoder shown in FIGS. 1A, 1B does not face a problem when image data is reproduced in the normal-speed positive direction playback and in the still playback, i.e., such digital VCR can reproduce recorded data by decoding recorded data correctly. However, such digital VCR are facing a serious problem when image data is reproduced in the normal-speed reverse direction playback mode, the positive direction playback at other speed than the normal speed and in the reverse direction high-speed playback mode.
Initially, the problem that the digital VCR encounters in the normal-speed reverse direction playback mode and the reverse direction playback mode at other speed than the normal speed will be described. A manner in which such a problem is caused will be described with reference to the drawings.
FIG. 5 shows examples of frame image data obtained when the digital VCR reproduces recorded data recorded on a magnetic tape in the reverse direction playback mode while a capstan motor is driven in the reverse direction.
As shown in FIG. 5, because image data is reproduced in the reverse direction playback mode, the arrangement of frame image data is reversed to that of frame image data shown in FIG. 3. However, the order of frame image data provided within the GOP is the same. Even when this order is reversed, upon decoding, frame image data are constantly processed at the GOP unit.
In this reverse direction playback mode, frame image data are reproduced in the sequential order of frame image data I9 of frame number F9, frame image data B10 (or P10) of frame number F10 (or P10), frame image data I7 of frame number F7, frame image data B8 (or P8) of frame number F8, frame image data I5 of frame number F5, frame image data B6 (or P6) of frame number F6, frame image data I3 of frame number F3, frame image data B4 (or P4) of frame number F4, frame image data I1 of frame number F1 and frame image data B2 (or P2) of frame number F2.
In this case, frame image data I9, I7, I5, I3 and I1 can be decoded without any trouble because they are intraframe-encoded upon encoding. Therefore, if they are decoded, then a satisfactory image can be obtained without trouble.
On the other hand, the frame image data B10 is decoded by use of frame image data I9 and the frame image data I7 as shown by arrows in FIG. 5; the frame image data B8 is decoded by use of the frame image data I7 and the frame image data Is as shown by arrows in FIG. 5; the frame image data B6 is decoded by use of the frame image data I5 and the frame image data I3 as shown by arrows in FIG. 5; the frame image data B4 is decoded by use of the frame image data I3 and the frame image data I1 as shown by arrows in FIG. 5; and the frame image data B2 is decoded by use of only the frame image data I1 as shown by an arrow in FIG. 5.
The frame image data I1, I3, I5, I7 and I9 used when the frame image data B2, B4, B6, B8 and B10 are encoded and decoded are compared with each other. The correct patterns are such ones used upon encoding as is clear from the comparison of FIGS. 3 and 5. In FIG. 5, the patterns different from those of FIG. 3 are marked with "X".
As shown in FIG. 3, upon encoding, a frame image data B10 is encoded by use of frame image data 19 and frame image data I11, not shown. On the other hand, as shown in FIG. 5, the frame image data B10 is decoded by use of frame image data I9 and frame image data I7 upon decoding. Therefore, as shown by "X" in FIG. 5, the use of the frame image data I7 upon decoding is not correct.
As shown in FIG. 3, the frame image data B8 is encoded by use of frame image data I7 and frame image data I9 upon encoding. On the other hand, as shown in FIG. 5, the frame image data B8 is decoded by use of frame image data I7 and frame image data I5. Therefore, as shown by "X" in FIG. 5, the use of the frame image I5 upon decoding is not correct.
As shown in FIG. 3, the frame image data B6 is encoded by use of frame image data I5 and frame image data I7 upon encoding. On the other hand, as shown in FIG. 5, the frame image data B6 is decoded by use of frame image data I5 and frame image data I3 upon decoding. Therefore, as shown by "X" in FIG. 5, the use of frame image data I3 upon decoding is not correct.
As shown in FIG. 3, the frame image data B4 is encoded by use of frame image data I3 and frame image data I5 upon encoding. On the other hand, as shown in FIG. 5, the frame image data B4 is decoded by use of frame image data I3 and frame image data I1 upon decoding. Therefore, as shown by "X" in FIG. 5, the use of the frame image data I1 upon decoding is not correct.
As shown in FIG. 3, the frame image data B2 is encoded by use of frame image data I1 and frame image data I3 upon encoding. On the other hand, as shown in FIG. 5, the frame image data B2 is decoded by use of only frame image data I1 upon decoding. Therefore, as shown by "X" in FIG. 5, there is not provided one frame image data that should be used upon decoding.
As is clear from the above description, when the magnetic tape is reproduced in the reverse direction, the same frame image data as those used in the encoding cannot be used and an incorrect processing is executed with the result that a picture quality of a reproduced image is deteriorated considerably. In the case of P picture, if frame image data of succeeding frame is used upon encoding, there then arises a similar problem.
Problems caused when the magnetic tape is reproduced at speed other than the normal speed in the positive or reverse direction will be described below. As is conventional, when recorded data recorded on the magnetic tape is reproduced, it becomes difficult for the playback head 10 shown in FIG. 1B to accurately reproduce recorded data that is recorded on the magnetic tape so as to form oblique tracks. As a result, a drop-out of data is increased. Furthermore, if the reproducing speed is very much higher than the normal speed, there then occurs a problem that reproduced record data cannot be decoded satisfactorily.
Therefore, in the recording and reproducing apparatus using the video encoder and the video decoder conforming to the storage moving image coding standard, e.g., the digital VCR shown in FIGS. 1A, 1B, in the playback mode other than the playback mode where the magnetic tape is reproduced at normal speed in the positive direction, there are used only frame image data serving as I pictures obtained by intraframe-encoding. In the digital VCR using the video encoder and the video decoder conforming to the storage moving image encoding standard based on MPEG2, for example, one GOP consists of 15 frames, and of frame image data of 15 frames comprising one GOP, there is provided one I picture, and all remaining frame image data become B pictures and P pictures.
Accordingly, when only the I picture is used, only one frame is reproduced per 15 frames, and a picture quality of a reproduced image becomes excellent. However, images of 14 frames cannot be reproduced per 15 frames, and hence a motion of a moving object within the image cannot be made smooth. To avoid this shortcoming, it is proposed that the number of I pictures contained within one GOP is increased. However, the number of B pictures and P pictures which are difference images is decreased, and a data amount of encoded data in one GOP is increased, which is not a good idea from an encoding efficiency standpoint.
In short, in the playback modes other than the playback mode where the magnetic tape is reproduced at normal speed in the positive direction, if it is intended that a picture quality is improved much more and that a motion of a moving object within the image is made smooth, there is then the disadvantage that the encoding efficiency is deteriorated.
While one GOP consists of 2 frames as shown in FIGS. 3 and 5 for convenience sake of explanation, if one GOP consists of frame image data of 15 frames as earlier noted, then upon edit, the edit unit always becomes 15 frames. The reason for this is that, unless the edit unit is GOP which is the encoding and decoding unit, then image data used for edit cannot be obtained. If somewhere of GOP is used as an edit point, then pictures ahead of or behind the edit point cannot be reproduced correctly because there is no image data which precedes the edit point from a time standpoint. Furthermore, since it is customary that the user determines the edit point with reference to a reproduced picture, it is difficult for the user to detect a position at which a picture displayed on a television monitor is recorded on the magnetic tape. Accordingly, it is difficult for the user to enter time code data in a manual fashion, which disadvantage is desired to be improved.